Controlling Optical Parameters at a User&#39;s Eye

ABSTRACT

Systems and methods dynamically control optical conditions presented to a user by an artificial reality system according to monitored visual experience parameters for the user. For example, the artificial reality presentation to the user can be tracked to monitor visual experience parameters, such as light characteristics (e.g., color), focal distances, virtual object characteristics (e.g., objects/text color, size, etc.), aggregated defocus distance of background, luminance, activity, eye movement, accommodation distances, and other suitable conditions. Implementations can vary optical conditions presented/displayed by the artificial reality system according to the monitoring by altering the focal distance for virtual objects, text size, text/background color, light characteristics, and other suitable optical conditions. In some examples, user preferences for optical conditions can be determined according to the monitoring. Implementations can vary the optical conditions to be different from those defined by the user preferences to provide an intervention and mitigate against undesirable impacts.

PRIORITY

This application claims priority to U.S. Provisional Patent ApplicationNo. 63/344,385 titled “CONTROLLING OPTICAL PARAMETERS AT A USER'S EYE,”filed May 20, 2022, with attorney docket No. 3589-0141PV01, U.S.Provisional Patent Application No. 63/344,389 titled “MODEL BASEDCONTROL OF OPTICAL PARAMETERS AT A USER'S EYE,” filed May 20, 2022, withattorney docket No. 3589-0141PV02, U.S. Provisional Patent ApplicationNo. 63/344,393 titled “AUTOMATIC VARIED VISUAL EXPERIENCE IN ANARTIFICIAL REALITY SYSTEM,” filed May 20, 2022, with attorney docket No.3589-0141 PV03, and U.S. Provisional Patent Application No. 63/384,258titled “MODEL BASED CONTROL OF OPTICAL PARAMETERS AT A USER'S EYE,”filed Nov. 18, 2022, with attorney docket No. 3589-0141 PV04, which areeach herein incorporated by reference in their entireties.

TECHNICAL FIELD

The present disclosure is directed to controlling optical parameters ata user's eye e.g., using an artificial reality system and tracked userconditions.

BACKGROUND

Artificial reality devices have grown in popularity with users, and thisgrowth is predicted to accelerate. These devices can immerse a user inan artificial reality environment and display objects within thisimmersive experience. For example, in an augmented reality environmentor mixed reality environment, portions of the environment can correspondto a real-world setting, such as a room, object, background, etc. Theartificial reality device can add one or more virtual objects to thisenvironment that do not correspond with the real-world setting. Inanother example, the immersive experience can be a virtual realityenvironment with one or more virtual objects. A user's eyes perceiveobject at specific locations in these immersive environments accordingto light from the artificial reality device (e.g., light displayed bythe device, light that passes through the device, light manipulated byone or more lenses of the device, etc.). Accordingly, artificial realitydevices can impact a user's eyes, including impacts on eye fatigue, eyebiology, vision fidelity, or other suitable eye characteristics.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an overview of devices on whichsome implementations of the present technology can operate.

FIG. 2A is a wire diagram illustrating a virtual reality headset whichcan be used in some implementations of the present technology.

FIG. 2B is a wire diagram illustrating a mixed reality headset which canbe used in some implementations of the present technology.

FIG. 2C is a wire diagram illustrating controllers which, in someimplementations, a user can hold in one or both hands to interact withan artificial reality environment.

FIG. 3 is a block diagram illustrating an overview of an environment inwhich some implementations of the present technology can operate.

FIG. 4 is a block diagram illustrating components which, in someimplementations, can be used in a system employing the disclosedtechnology.

FIG. 5 is a conceptual diagram illustrating light entering a user's eyevia an artificial reality system.

FIG. 6 is a conceptual diagram illustrating an artificial realityenvironment with virtual objects.

FIG. 7 is a conceptual diagram illustrating variations of opticalconditions at a user's eye.

FIG. 8 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using an artificial reality system andtracked user conditions.

FIG. 9 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using an artificial reality system accordingto monitored temporal conditions.

FIG. 10 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using a personalized eye model.

FIG. 11 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling visualparameters by an artificial reality system to vary a user experience.

FIG. 12 is a diagram illustrating a model of the optical parameters oflight output from a display device.

FIG. 13 is a diagram illustrating another model of the opticalparameters of light output from another display device.

FIG. 14 is a diagram illustrating a model of the optical parameters oflight output from a display device in combination with an eye model.

FIG. 15 is a diagram illustrating another model of the opticalparameters of light output from another display device in combinationwith an eye model.

FIG. 16 is a diagram of simulated light exposed to an eye model.

FIG. 17 is a graph of optical characteristics simulated by exposinglight to an eye model.

FIG. 18 is another graph of optical characteristics simulated byexposing light to an eye model.

FIGS. 19 and 20 are graphs of optical characteristics simulated byexposing light to another eye model.

FIGS. 21 and 22 are graphs of optical characteristics simulated byexposing light to eye model variations.

The techniques introduced here may be better understood by referring tothe following Detailed Description in conjunction with the accompanyingdrawings, in which like reference numerals indicate identical orfunctionally similar elements.

DETAILED DESCRIPTION

Aspects of the present disclosure are directed to controlling opticalparameters at a user's eye using an artificial reality system andtracked user conditions. Implementations of the artificial realitysystem immerse a user in an artificial reality environment that includesone or more virtual objects and/or one or more real-world objects. Forexample, a head-mounted display of the artificial reality system can bepositioned to generate light that enters the user's eyes and/orpass-through light (e.g., selectively pass-through real-world light)that enters the user's eyes. By controlling and/or manipulating thelight from the system, the artificial reality system can present theimmersive environment to the user. Implementations can use any othersuitable optical system to generate the artificial reality environment.

Implementations of a light coordinator can control the light that entersone or more of the user's eyes (e.g., via the artificial reality system)according to tracked user conditions. For example, based on tracked usereye positioning, the light coordinator can adjust the light that entersthe user's eye to control an image shell generated at the user's eye. Animage shell refers to the way light, that enters the eye, focuses on theretina of the eye. Example properties of an image shell include imageshell centration, image shell curvature, image shell shape, etc. A usermay focus on an object in an artificial reality environment (e.g.,real-world object or virtual object) and light from the object cangenerate an image shell at the user's eyes. The image shell at theuser's eyes can impact the user's vision and/or eye biology. Otheroptical parameters/light properties can also impact the image shelland/or a user's vision, such as luminance, chromatic balance, lightspectrum (e.g., color), focal distance at retinal eccentricity,modulation transfer function at retinal eccentricity, etc.

Some implementations control optical parameters at a user's eye using aneye model and an artificial reality system. An eye model coordinator cancreate an eye model specific to a user based on user characteristics,such as spherical refraction of the user's eye(s), user age, axiallength, choroidal thickness, ocular curvature, phoria at near/far, orother suitable characteristics. The eye model can simulate opticalparameters generated at the user's eye(s) by light from the artificialreality system. For example, the light coordinator can use output fromthe eye model to control light provided to the user's eye(s) via theartificial reality system. In some implementations, user eye parameters,such as accommodative state, pupil size, pupil position in eye box, gazevector/fixation distance, vergence state/distance, etc., can be trackedand provided to the user eye model to simulate optical parameters at theuser's eye. In this example, variable input values (e.g., tracked eyeparameters) can be fed to the eye model over time and the lightcoordinator can dynamically control the light that enters the user'seye(s) via the artificial reality system using variable output from theeye model that changes over time.

Implementations of the light coordinator can adjust, based on trackeduser eye positioning and/or output from the eye model, any suitableaspect or characteristic of light that enters the user's eye to controlany suitable optical parameters that impact the user's vision. Forexample, the light coordinator can adjust/manipulate light to controloptical parameters by varying focal distance at eccentricity, varyingimage shell curvature, varying resolution/modulation transfer function(MTF) at eccentricity (e.g., between colors), varying field of view,varying focal distance, varying peripheral light curvature, varying thecentration of light curvature, adding peripheral myopic defocus cue(s)in a user's periphery, reducing or dynamically varying peripheralcontrast, and controlling other suitable optical parameters.Implementations of the light coordinator can monitor dynamic parameters(e.g., while the user is immersed in an XR environment) and generate thepersonalized XR presentation/display using the personalized eye model.

In some implementations, the light coordinator and eye model candynamically control optical conditions (e.g., optical conditions at theuser's eye, optical conditions presented/displayed by the XR system),such as focal distance, field curvature, distance, or a combination (perpixel/area), locus of centration of focus/curvature, depth of field,resolution/modulation transfer function (MTF) at eccentricity (e.g.,optical, content enhancement, and/or rendered via longitudinal chromaticaberration), spectrum (e.g., RGB primary wavelength/bandwidth, balancebetween), contrast polarity (e.g., white text on black vs. black text onwhite), and other suitable optical conditions. In some implementations,based on tracked user eye positioning and/or outputs from thepersonalized model, the light coordinator can adjust the light thatenters the user's eye to control an image shell generated at the user'seye.

Some implementations of the light coordinator can control the light thatenters one or more of the user's eyes (e.g., via the artificial realitysystem) according to any suitable temporal condition. For example, auser's circadian rhythm can be impacted by the light that enters theuser's eye via the artificial reality system. The light coordinator cancontrol the characteristics of the light that enters the user's eye tosync with and/or mitigate interference with the user's circadian rhythm.In another example, some users may be more or less sensitive to certaintypes of light at different times of day. The light coordinator cancontrol the characteristics of the light that enters the user's eyeaccording to the time of day. Implementations of the light coordinatorcan control the characteristics of the light that enters the user's eyeaccording to any other suitable temporal condition.

Implementations of a light coordinator can control the light that entersone or more of the user's eyes (e.g., via the artificial reality system)according to a tracked user experience. For example, parameters forvisual experiences presented to the user by the artificial realitysystem (e.g., an artificial reality environment visual experience) canbe monitored over time. Implementations of the light coordinator can,based on the monitored visual experience parameters, vary contentdisplayed by the artificial reality system in the artificial realityenvironment and/or manipulate light that enters the user's eye tocontrol optical parameters at the user's eye.

Example tracked visual experience parameters for the user includebackground characteristics (e.g., patterns, colors, distance, aggregateddefocus distance, etc.), foreground characteristics (e.g., object shape,dimensions, colors, distance, etc.), eye image shell characteristics(e.g., image shell curvature, centration, etc.), peripherycharacteristics (e.g., periphery curvature, contrast, defocus cues,etc.), focal distance, fixation distance, luminance, spectrum,Illumination wave shape/duty cycle, chromatic balance, contrast (e.g.,contrast polarity), spatial frequency, longitudinal chromatic aberration(LCA) at distance, field of view presented to the user, visual activity,eye movement, accommodation distances, and any other suitable visualexperience parameters.

Implementations of the light coordinator can vary the user's experienceaccording to the monitoring. For example, background distances,patterns, and colors can be varied over time. In another example, anobject in the foreground can have varied dimensions, shape, color,distances, etc. In another example, light can be adjusted (e.g., usingone or more lenses) to control the image shell created at the user's eyesuch that the curvature of the image shell and/or centration of theimage shell is varied over time. In another example, the focal distanceand/or fixation distance experienced by a user can be varied over time.In another example, peripheral characteristics experienced by the user,such as periphery curvature, contrast, and/or defocus cues can be variedover time. In another example, movement of images/objects displayed bythe artificial reality system can be varied according to monitored eyemovement/visual activity.

In another example, the rendered longitudinal chromatic aberration (LCA)experienced by a user can be varied over time. In another example, thespatial frequency (e.g., scene contrast at spatial frequency)experienced by a user can be varied over time. In another example, thecontrast polarity experienced by a user can be varied over time. Inanother example, the Illumination wave shape/duty cycle experienced by auser can be varied over time. In another example, the luminance,spectrum, and/or chromatic balance experienced by a user can be variedover time.

In some implementations, a user may prefer a given object distance, textsize, or other suitable visual parameter, and the user may curate the XRenvironment to accommodate these user preferences. However, extendedexposure to certain visual experiences may have an undesirable impact,such as eye fatigue, improper eye health practices, or other suitableundesirable impacts. Implementations of the light coordinator can varycontent and or optics presented/displayed by the XR system, such as byaltering distance for virtual objects, text size, text/background color,light characteristics, and other suitable visual experience parameters.In some examples, user preferences for visual experiences can bedetermined according to monitored visual experience parameters, wherefuture optical conditions are expected to comply with the determineduser preferences without intervention. Implementations of the lightcoordinator can vary content and/or optics to be different from thosedefined by the user preferences to provide such an intervention andmitigate against undesirable impacts.

In some implementations, the light coordinator can compare the monitoredvisual experience parameter(s) to one or more criteria and adjustcontent and/or manipulate light that enters the user's eye to varyvisual experience parameter(s) when the one or more criteria are met.The implemented variations can improve the user's experience with the XRsystem. For example, varying one or more of the monitored visualexperience parameters can reduce eye fatigue or improve eye biology,visual acuity, and other aspects of user eyesight.

Implementations can manipulate the light that enters the user's one ormore eyes (e.g., tracked pupil location) via the artificial realitysystem using one or more devices. For example, a gradient-index (GRIN)liquid crystal (LC) device with a patterned electrode can be used toselectively manipulate light that passes through the device. Bycontrolling individual portions of the GRIN LC device, the light thatenters a user's tracked pupil location can be manipulated to controloptical parameters at the user's eye. For example, the opticalproperties of a GRIN LC lens (e.g., optical power) can be selectivelytunable by applied electric fields. Implementations can apply electricfields and selectively tune portions of the GRIN LC device according toa user's tracked pupil location to manipulate light that enters theuser's pupil and control optical parameters at the user's eye. U.S.patent application Ser. No. 17/173,177, filed Feb. 10, 2021, which ishereby incorporated by reference in its entirety, discloses examples ofGRIN LC devices that can be used in some implementations.

In another example, a freeform varifocal optical assembly that includesan optical stack can be configured to enhance optical parameters at auser's eye, such as accommodation of one or both eyes. An examplefreeform varifocal optical assembly includes Pancharatnam-Berry phase(PBP) lenses, PBP gratings, polarization sensitive hologram (PSH)lenses, PSH gratings, metamaterials, or combinations thereof. Thefreeform varifocal optical assembly can be used to output an adjustedwaveform from an arbitrary input waveform (independent of the opticalsystem of a display device). The optical modules can also include aplurality of switchable polarization control optical components, such asa switchable retarder. By including a plurality of polarizationsensitive lensing elements having different optical powers andcontrolling the plurality of switchable polarization control opticalcomponents (and the polarization sensitive lensing elements), thefreeform varifocal optical system can provide focal power and aberrationcompensation. Implementations can adjust the switchable components ofthe freeform varifocal optical assembly to manipulate light that passesthrough the assembly and control optical parameters at the user's eye(e.g., according to a tracked pupil location). U.S. patent applicationSer. No. 16/854,528, filed Apr. 21, 2020, which is hereby incorporatedby reference, discloses examples of a freeform varifocal opticalassembly that can be used in some implementations.

In yet another example, a varifocal optical system includes a pluralityof optical elements, and at least some of the optical elements include acontrollable focal power. For example, the varifocal optical system mayinclude a plurality of lenses or gratings formed from liquid crystals,such as Pancharatnam-Berry Phase (PBP; also referred to as geometricphase) lenses, PBP gratings (also referred to as geometric phasegratings), polarization sensitive hologram (PSH) lenses, PSH gratings,and/or liquid crystal optical phase arrays. By controlling polarizationof light incident on each respective lens or grating, and/or a state ofthe lens or grating, the optical system may be controlled to have aselected total optical power. Implementations can adjust thepolarization of light incident on each respective lens or grating and/ora state of the lens or grating to adjust the total optical power of thesystem and control optical parameters at the user's eye. U.S. patentapplication Ser. No. 16/723,152, filed Dec. 20, 2019, which is herebyincorporated by reference, discloses examples of varifocal opticalsystems that can be used in some implementations.

In a further example, a steered retinal display device can be used byimplementations to control optical parameters at a user's eye bymanipulating displayed pixels. For example, a steered retinal projectionsystem can create an image on the retina that tracks with eye movement.The optical axis and image plane (optionally) can be steered accordingto eye/gaze tracking so that the angular, lateral and axial placement ofthe imaging forming light is both located properly in the eye as well asimage corrected according to potential changes caused by steering. In anexample, steering can be performed using combinations of conventionalcomponents and/or techniques that utilize mirrors or by other techniquessuch as optical components that change angle or displace throughreflection, refraction, diffraction, etc. Such techniques may useelectrically changed index of refraction, dispersion caused byillumination wavelength shift, polarization change, and/or electricallycontrolled mechanical motions.

Embodiments of the disclosed technology may include or be implemented inconjunction with an artificial reality system. Artificial reality orextra reality (XR) is a form of reality that has been adjusted in somemanner before presentation to a user, which may include, e.g., virtualreality (VR), augmented reality (AR), mixed reality (MR), hybridreality, or some combination and/or derivatives thereof. Artificialreality content may include completely generated content or generatedcontent combined with captured content (e.g., real-world photographs).The artificial reality content may include video, audio, hapticfeedback, or some combination thereof, any of which may be presented ina single channel or in multiple channels (such as stereo video thatproduces a three-dimensional effect to the viewer). Additionally, insome embodiments, artificial reality may be associated withapplications, products, accessories, services, or some combinationthereof, that are, e.g., used to create content in an artificial realityand/or used in (e.g., perform activities in) an artificial reality. Theartificial reality system that provides the artificial reality contentmay be implemented on various platforms, including a head-mounteddisplay (HMD) connected to a host computer system, a standalone HMD, amobile device or computing system, a “cave” environment or otherprojection system, or any other hardware platform capable of providingartificial reality content to one or more viewers.

“Virtual reality” or “VR,” as used herein, refers to an immersiveexperience where a user's visual input is controlled by a computingsystem. “Augmented reality” or “AR” refers to systems where a user viewsimages of the real world after they have passed through a computingsystem. For example, a tablet with a camera on the back can captureimages of the real world and then display the images on the screen onthe opposite side of the tablet from the camera. The tablet can processand adjust or “augment” the images as they pass through the system, suchas by adding virtual objects. “Mixed reality” or “MR” refers to systemswhere light entering a user's eye is partially generated by a computingsystem and partially composes light reflected off objects in the realworld. For example, a MR headset could be shaped as a pair of glasseswith a pass-through display, which allows light from the real world topass through a waveguide that simultaneously emits light from aprojector in the MR headset, allowing the MR headset to present virtualobjects intermixed with the real objects the user can see. “Artificialreality,” “extra reality,” or “XR,” as used herein, refers to any of VR,AR, MR, or any combination or hybrid thereof.

Several implementations are discussed below in more detail in referenceto the figures. FIG. 1 is a block diagram illustrating an overview ofdevices on which some implementations of the disclosed technology canoperate. The devices can comprise hardware components of a computingsystem 100 that control light in an artificial reality system accordingto tracked user conditions. In various implementations, computing system100 can include a single computing device 103 or multiple computingdevices (e.g., computing device 101, computing device 102, and computingdevice 103) that communicate over wired or wireless channels todistribute processing and share input data. In some implementations,computing system 100 can include a stand-alone headset capable ofproviding a computer created or augmented experience for a user withoutthe need for external processing or sensors. In other implementations,computing system 100 can include multiple computing devices such as aheadset and a core processing component (such as a console, mobiledevice, or server system) where some processing operations are performedon the headset and others are offloaded to the core processingcomponent. Example headsets are described below in relation to FIGS. 2Aand 2B. In some implementations, position and environment data can begathered only by sensors incorporated in the headset device, while inother implementations one or more of the non-headset computing devicescan include sensor components that can track environment or positiondata.

Computing system 100 can include one or more processor(s) 110 (e.g.,central processing units (CPUs), graphical processing units (GPUs),holographic processing units (HPUs), etc.) Processors 110 can be asingle processing unit or multiple processing units in a device ordistributed across multiple devices (e.g., distributed across two ormore of computing devices 101-103).

Computing system 100 can include one or more input devices 120 thatprovide input to the processors 110, notifying them of actions. Theactions can be mediated by a hardware controller that interprets thesignals received from the input device and communicates the informationto the processors 110 using a communication protocol. Each input device120 can include, for example, a mouse, a keyboard, a touchscreen, atouchpad, a wearable input device (e.g., a haptics glove, a bracelet, aring, an earring, a necklace, a watch, etc.), a camera (or otherlight-based input device, e.g., an infrared sensor), a microphone, orother user input devices.

Processors 110 can be coupled to other hardware devices, for example,with the use of an internal or external bus, such as a PCI bus, SCSIbus, or wireless connection. The processors 110 can communicate with ahardware controller for devices, such as for a display 130. Display 130can be used to display text and graphics. In some implementations,display 130 includes the input device as part of the display, such aswhen the input device is a touchscreen or is equipped with an eyedirection monitoring system. In some implementations, the display isseparate from the input device. Examples of display devices are: an LCDdisplay screen, an LED display screen, a projected, holographic, oraugmented reality display (such as a heads-up display device or ahead-mounted device), and so on. Other I/O devices 140 can also becoupled to the processor, such as a network chip or card, video chip orcard, audio chip or card, USB, firewire or other external device,camera, printer, speakers, CD-ROM drive, DVD drive, disk drive, etc.

In some implementations, input from the I/O devices 140, such ascameras, depth sensors, IMU sensor, GPS units, LiDAR or othertime-of-flights sensors, etc. can be used by the computing system 100 toidentify and map the physical environment of the user while tracking theuser's location within that environment. This simultaneous localizationand mapping (SLAM) system can generate maps (e.g., topologies, girds,etc.) for an area (which may be a room, building, outdoor space, etc.)and/or obtain maps previously generated by computing system 100 oranother computing system that had mapped the area. The SLAM system cantrack the user within the area based on factors such as GPS data,matching identified objects and structures to mapped objects andstructures, monitoring acceleration and other position changes, etc.

Computing system 100 can include a communication device capable ofcommunicating wirelessly or wire-based with other local computingdevices or a network node. The communication device can communicate withanother device or a server through a network using, for example, TCP/IPprotocols. Computing system 100 can utilize the communication device todistribute operations across multiple network devices.

The processors 110 can have access to a memory 150, which can becontained on one of the computing devices of computing system 100 or canbe distributed across of the multiple computing devices of computingsystem 100 or other external devices. A memory includes one or morehardware devices for volatile or non-volatile storage, and can includeboth read-only and writable memory. For example, a memory can includeone or more of random access memory (RAM), various caches, CPUregisters, read-only memory (ROM), and writable non-volatile memory,such as flash memory, hard drives, floppy disks, CDs, DVDs, magneticstorage devices, tape drives, and so forth. A memory is not apropagating signal divorced from underlying hardware; a memory is thusnon-transitory. Memory 150 can include program memory 160 that storesprograms and software, such as an operating system 162, lightcoordinator 164, and other application programs 166. Memory 150 can alsoinclude data memory 170 that can include, e.g., users' health data, usereye data, user preferences, historical user visual experiences,configuration data, settings, user options or preferences, etc., whichcan be provided to the program memory 160 or any element of thecomputing system 100.

Some implementations can be operational with numerous other computingsystem environments or configurations. Examples of computing systems,environments, and/or configurations that may be suitable for use withthe technology include, but are not limited to, XR headsets, personalcomputers, server computers, handheld or laptop devices, cellulartelephones, wearable electronics, gaming consoles, tablet devices,multiprocessor systems, microprocessor-based systems, set-top boxes,programmable consumer electronics, network PCs, minicomputers, mainframecomputers, distributed computing environments that include any of theabove systems or devices, or the like.

FIG. 2A is a wire diagram of a virtual reality head-mounted display(HMD) 200, in accordance with some embodiments. The HMD 200 includes afront rigid body 205 and a band 210. The front rigid body 205 includesone or more electronic display elements of an electronic display 245, aninertial motion unit (IMU) 215, one or more position sensors 220,locators 225, and one or more compute units 230. The position sensors220, the IMU 215, and compute units 230 may be internal to the HMD 200and may not be visible to the user. In various implementations, the IMU215, position sensors 220, and locators 225 can track movement andlocation of the HMD 200 in the real world and in an artificial realityenvironment in three degrees of freedom (3DoF) or six degrees of freedom(6DoF). For example, the locators 225 can emit infrared light beamswhich create light points on real objects around the HMD 200. As anotherexample, the IMU 215 can include e.g., one or more accelerometers,gyroscopes, magnetometers, other non-camera-based position, force, ororientation sensors, or combinations thereof. One or more cameras (notshown) integrated with the HMD 200 can detect the light points. Computeunits 230 in the HMD 200 can use the detected light points toextrapolate position and movement of the HMD 200 as well as to identifythe shape and position of the real objects surrounding the HMD 200.

The electronic display 245 can be integrated with the front rigid body205 and can provide image light to a user as dictated by the computeunits 230. In various embodiments, the electronic display 245 can be asingle electronic display or multiple electronic displays (e.g., adisplay for each user eye). Examples of the electronic display 245include: a liquid crystal display (LCD), an organic light-emitting diode(OLED) display, an active-matrix organic light-emitting diode display(AMOLED), a display including one or more quantum dot light-emittingdiode (QOLED) sub-pixels, a projector unit (e.g., microLED, LASER,etc.), some other display, or some combination thereof.

In some implementations, the HMD 200 can be coupled to a core processingcomponent such as a personal computer (PC) (not shown) and/or one ormore external sensors (not shown). The external sensors can monitor theHMD 200 (e.g., via light emitted from the HMD 200) which the PC can use,in combination with output from the IMU 215 and position sensors 220, todetermine the location and movement of the HMD 200.

FIG. 2B is a wire diagram of a mixed reality HMD system 250 whichincludes a mixed reality HMD 252 and a core processing component 254.The mixed reality HMD 252 and the core processing component 254 cancommunicate via a wireless connection (e.g., a 60 GHz link) as indicatedby link 256. In other implementations, the mixed reality system 250includes a headset only, without an external compute device or includesother wired or wireless connections between the mixed reality HMD 252and the core processing component 254. The mixed reality HMD 252includes a pass-through display 258 and a frame 260. The frame 260 canhouse various electronic components (not shown) such as light projectors(e.g., LASERs, LEDs, etc.), cameras, eye-tracking sensors, MEMScomponents, networking components, etc.

The projectors can be coupled to the pass-through display 258, e.g., viaoptical elements, to display media to a user. The optical elements caninclude one or more waveguide assemblies, reflectors, lenses, mirrors,collimators, gratings, etc., for directing light from the projectors toa user's eye. Image data can be transmitted from the core processingcomponent 254 via link 256 to HMD 252. Controllers in the HMD 252 canconvert the image data into light pulses from the projectors, which canbe transmitted via the optical elements as output light to the user'seye. The output light can mix with light that passes through the display258, allowing the output light to present virtual objects that appear asif they exist in the real world.

Similarly to the HMD 200, the HMD system 250 can also include motion andposition tracking units, cameras, light sources, etc., which allow theHMD system 250 to, e.g., track itself in 3DoF or 6DoF, track portions ofthe user (e.g., hands, feet, head, or other body parts), map virtualobjects to appear as stationary as the HMD 252 moves, and have virtualobjects react to gestures and other real-world objects.

FIG. 2C illustrates controllers 270 (including controller 276A and276B), which, in some implementations, a user can hold in one or bothhands to interact with an artificial reality environment presented bythe HMD 200 and/or HMD 250. The controllers 270 can be in communicationwith the HMDs, either directly or via an external device (e.g., coreprocessing component 254). The controllers can have their own IMU units,position sensors, and/or can emit further light points. The HMD 200 or250, external sensors, or sensors in the controllers can track thesecontroller light points to determine the controller positions and/ororientations (e.g., to track the controllers in 3DoF or 6DoF). Thecompute units 230 in the HMD 200 or the core processing component 254can use this tracking, in combination with IMU and position output, tomonitor hand positions and motions of the user. The controllers can alsoinclude various buttons (e.g., buttons 272A-F) and/or joysticks (e.g.,joysticks 274A-B), which a user can actuate to provide input andinteract with objects.

In various implementations, the HMD 200 or 250 can also includeadditional subsystems, such as an eye tracking unit, an audio system,various network components, etc., to monitor indications of userinteractions and intentions. For example, in some implementations,instead of or in addition to controllers, one or more cameras includedin the HMD 200 or 250, or from external cameras, can monitor thepositions and poses of the user's hands to determine gestures and otherhand and body motions. As another example, one or more light sources canilluminate either or both of the user's eyes and the HMD 200 or 250 canuse eye-facing cameras to capture a reflection of this light todetermine eye position (e.g., based on set of reflections around theuser's cornea), modeling the user's eye and determining a gazedirection.

FIG. 3 is a block diagram illustrating an overview of an environment 300in which some implementations of the disclosed technology can operate.Environment 300 can include one or more client computing devices 305A-D,examples of which can include computing system 100. In someimplementations, some of the client computing devices (e.g., clientcomputing device 305B) can be the HMD 200 or the HMD system 250. Clientcomputing devices 305 can operate in a networked environment usinglogical connections through network 330 to one or more remote computers,such as a server computing device.

In some implementations, server 310 can be an edge server which receivesclient requests and coordinates fulfillment of those requests throughother servers, such as servers 320A-C. Server computing devices 310 and320 can comprise computing systems, such as computing system 100. Thougheach server computing device 310 and 320 is displayed logically as asingle server, server computing devices can each be a distributedcomputing environment encompassing multiple computing devices located atthe same or at geographically disparate physical locations.

Client computing devices 305 and server computing devices 310 and 320can each act as a server or client to other server/client device(s).Server 310 can connect to a database 315. Servers 320A-C can eachconnect to a corresponding database 325A-C. As discussed above, eachserver 310 or 320 can correspond to a group of servers, and each ofthese servers can share a database or can have their own database.Though databases 315 and 325 are displayed logically as single units,databases 315 and 325 can each be a distributed computing environmentencompassing multiple computing devices, can be located within theircorresponding server, or can be located at the same or at geographicallydisparate physical locations.

Network 330 can be a local area network (LAN), a wide area network(WAN), a mesh network, a hybrid network, or other wired or wirelessnetworks. Network 330 may be the Internet or some other public orprivate network. Client computing devices 305 can be connected tonetwork 330 through a network interface, such as by wired or wirelesscommunication. While the connections between server 310 and servers 320are shown as separate connections, these connections can be any kind oflocal, wide area, wired, or wireless network, including network 330 or aseparate public or private network.

FIG. 4 is a block diagram illustrating components 400 which, in someimplementations, can be used in a system employing the disclosedtechnology. Components 400 can be included in one device of computingsystem 100 or can be distributed across multiple of the devices ofcomputing system 100. The components 400 include hardware 410, mediator420, and specialized components 430. As discussed above, a systemimplementing the disclosed technology can use various hardware includingprocessing units 412, working memory 414, input and output devices 416(e.g., cameras, displays, IMU units, network connections, etc.), andstorage memory 418. In various implementations, storage memory 418 canbe one or more of: local devices, interfaces to remote storage devices,or combinations thereof. For example, storage memory 418 can be one ormore hard drives or flash drives accessible through a system bus or canbe a cloud storage provider (such as in storage 315 or 325) or othernetwork storage accessible via one or more communications networks. Invarious implementations, components 400 can be implemented in a clientcomputing device such as client computing devices 305 or on a servercomputing device, such as server computing device 310 or 320.

Mediator 420 can include components which mediate resources betweenhardware 410 and specialized components 430. For example, mediator 420can include an operating system, services, drivers, a basic input outputsystem (BIOS), controller circuits, or other hardware or softwaresystems.

Specialized components 430 can include software or hardware configuredto perform operations for controlling light using an artificial realitysystem according to tracked user conditions. Specialized components 430can include eye tracker 434, temporal monitor 436, dynamic state tracker438, eye model(s) 440, user experience monitor 442, content controller444, optics controller 446, and components and APIs which can be usedfor providing user interfaces, transferring data, and controlling thespecialized components, such as interfaces 432. In some implementations,components 400 can be in a computing system that is distributed acrossmultiple computing devices or can be an interface to a server-basedapplication executing one or more of specialized components 430.Although depicted as separate components, specialized components 430 maybe logical or other nonphysical differentiations of functions and/or maybe submodules or code-blocks of one or more applications.

Eye tracker 434 can track the movement of one or more of a user's eyes.For example, eye tracker 434 can track the location of a pupil of auser's eye within the eye box. In some implementations, eye tracker 434can track both the location of a user's eye and the user's gaze(according to head positioning). Eye tracker 434 can track the user'seye/head movement using one or more sensors, such as one or multiplecameras. In some implementations, the visual data can be processed byone or more machine learning models trained to perform eye/gazetracking. The machine learning models can be a trained neural network(e.g., a LSTM, convolutional network, or other network), or any othersuitable machine learning model configured to track user eye movements.Additional details on eye tracker 434 are provided below in relation toblocks 804 of FIG. 8 .

Temporal monitor 436 can monitor temporal conditions for a user. Thetemporal conditions can include the time of day, day of week, day of themonth, season of the year, user sleep cycle, user activity level, typeof user activity, or any other suitable temporal condition. In someimplementations, a sensor (e.g., wearable sensor, accelerometer,gyroscope, etc.) can provide user data to temporal monitor 436, such asthe user's steps, activity level, sleep cycle, and the like. Temporalmonitor 436 can include clock and calendar functionality (or can receiveclock and calendar data) to monitor temporal conditions for the user.Additional details on temporal monitor 436 are provided below inrelation to blocks 902, 904, 908, and 912 of FIG. 9 .

Dynamic state tracker 438 can track the movement of one or more of auser's eyes and other suitable dynamic states for implementations of theeye model. Example dynamic state parameters tracked by dynamic statetracker 438 include pupil size, pupil position in an eye box, gazevector/fixation distance, accommodative state, vergence state/distance,time of day, recent time of fixation at distance, background visualdistance/intensity (e.g., for AR/MR), and other suitable dynamic stateparameters.

Dynamic state tracker 438 can track the user's eye/head using one ormore sensors, such as one or multiple cameras. In some implementations,the visual data can be processed by one or more machine learning modelstrained to perform eye tracking, gaze tracking, eye monitoring, or othersuitable tasks. The machine learning models can be a trained neuralnetwork (e.g., a LSTM, convolutional network, or other network), or anyother suitable machine learning model configured to monitor user eyeparameters. In some implementations, dynamic state tracker 438 caninclude clock/calendar functionality (or can receive clock and calendardata) to monitor temporal conditions for the user. Additional details ondynamic state tracker 438 are provided below in relation to blocks 1008of FIG. 10 .

Eye model(s) 440 can be any suitable 3D, algorithmic, or numerical modelthat maps user eye characteristics. For example, eye model(s) 440 can bea regression model, machine learning model, numerical algorithm, 3Dmodel generated from machine learning outputs, and/or any other suitablemodel or combination of models. Example user characteristic variablesfor eye model(s) 440 can include user age, visual optics/refractiveerror (e.g., spherical, across field), axial length, choroidalthickness, ocular curvature, phoria at near/far, and the like. In someimplementations, a personalized eye model 440 for a user can receive theuser's characteristics and output optical control parameters for theuser. In this example, the optical control parameters can be specificvalues or value ranges for one or more optical parameters experienced atthe user's eye, and the XR system can manipulate light that enters theuser's eye (e.g., via optics controller 440 and the monitored dynamicstate parameters) in accordance with the optical control parameters.

In some implementations, eye model(s) 440 can map user characteristicsand dynamic state parameters to optical parameter controls. For example,a personalized eye model 440 for a user can receive the user'scharacteristics and monitored dynamic state parameters (e.g., fromdynamic state tracker 438) for the user, and output optical controlparameters. In this example, the optical control parameters can bespecific values or value ranges for one or more optical parametersexperienced at the user's eye, and the XR system can manipulate lightthat enters the user's eye (e.g., via optics controller 446) inaccordance with the optical control parameters. Example dynamic stateparameter variables for eye model(s) 440 can include pupil size, pupilposition (e.g., in an eye box or as mapped by a 3D model of the user'seye(s)), gaze vector/fixation distance, accommodative state, vergencestate/distance, time of day, recent time of fixation at distance,background visual distance/intensity (e.g., for AR/MR), and othersuitable dynamic state inputs.

In some implementations, developing eye model(s) 440 can include manualdiagnostics and/or testing to determine optical tunings for users withparticular characteristics. In this example, eye model(s) 440 can beused to map particular user characteristics to optimal controlparameters for a user. In another example, one or more usercharacteristics (e.g., physical properties of the eye, axial length,etc.) can be used to generate a 3D model of the user's eye. Using the 3Dmodel and other characteristics for the XR system (e.g., lens distancefrom a user's eye, etc.) one or more light/display/lens conditions andresulting optical parameters at the user's eye can be simulated todetermine optimal control parameters. In some implementations, one ormore machine learning models can be trained/configured to receive, asinput, factors such as age, vision history (e.g., how many hours a daythe user looks at a screen), pupil size, and the like, and predict auser's eye characteristics (e.g., axial length, etc.). In this example,the predicted eye characteristics can be used to generate the 3D modelfor the user's eye and/or perform simulations to determine optimalcontrol parameters. Additional details on eye model(s) 440 are providedbelow in relation to blocks 1002 and 1044 of FIG. 10 .

User experience monitor 442 can monitor the visual experiences for auser, such as the parameters of the user's visual experiences with theXR system. For example, user experience monitor 442 can monitor the XRdisplay to detect background characteristics experienced by the userover time (e.g., patterns, colors, distance, aggregated defocusdistance, etc.); foreground characteristics experienced by the user overtime (e.g., object shape, dimensions, colors, distance, etc.);luminance, spectrum, and chromatic balance experienced by the user overtime; illumination wave shape/duty cycle experienced by the user overtime; contrast (e.g., contrast polarity) experienced by the user overtime; spatial frequency experienced by the user over time; longitudinalchromatic aberration (LCA) experienced by the user over time; and othersuitable visual experience parameters related to displayed content.

In some implementations, user experience monitor 442 can also monitoroptical parameters created at the user's eye by the XR system, such aseye image shell characteristics (e.g., image shell curvature,centration, etc.), periphery characteristics (e.g., periphery curvature,contrast, defocus cues, etc.), focal distance, eye movement,accommodation distances, and any other suitable optical parameters. Forexample, user experience monitor 442 and eye model(s) 440 can, incombination, simulate optical parameters created at the user's eye bythe XR system and monitor these simulated optical parameters over time.

User experience monitor 442 can monitor a user's visual experience overa defined duration of time, such as over a day, a week, a month, a year,and the like. For example, one or more visual experience profiles can bepopulated with the visual experience data monitored for a user, such asvisual experience profiles associated with different durations of time.Additional details on user experience monitor 442 are provided below inrelation to block 1104 of FIG. 11 .

Content controller 444 can control content displayed by the XR system,for example based on the monitored visual experience parameters. In someimplementations, a criteria can be defined, such as a criteria for anindividual monitored visual experience parameter or a combination ofmonitored visual experience parameters. When a particular visualexperience parameter related to content meets a defined criteria,content controller 444 can adjust the content displayed to a user by theXR system to vary the particular visual experience parameter. Forexample, a background characteristics criteria can define a thresholdperiod of time for a user to experience the same background features.When a monitored profile for the user indicates that the backgroundcharacteristics criteria has been met, content controller 444 can adjustthe background features to provide the user a varied experience (e.g.,adjust the background color, the distance, pattern, etc.). Othermonitored visual experience parameters can have defined criteria thattrigger adjustments to content to vary the visual experience parameters.Additional details on content controller 444 are provided below inrelation to block 1110 and 1112 of FIG. 11 .

Optics controller 446 can control light that enters the user's eye(s)via the XR system. For example, optics controller 446 can control one ormore optical devices (e.g., lenses) configured to manipulate light thatenters the user's eyes. In some implementations, physicalcharacteristics of the lenses can be adjusted by optics controller 446to control light that passes through the lenses. For example, electricfields can be selectively applied to a GRIN LC lens to selectively tuneportions of the lens, manipulate light that passes through the lens, andcontrol optical parameters at the user's eye. In another example,switchable components of a freeform varifocal optical assembly and/oroptical elements of a varifocal optical system with controllable focalpower can be selectively controlled to manipulate light that passesthrough the lens(es), and control optical parameters at the user's eye.

In another example, optics controller 446 can control a display thatemits light that enters the user's eye. In some implementations,characteristics of the emitted light, such as the luminance, spectrum,chromatic balance, and other suitable characteristics, can be controlledby optics controller 446. In some implementations, optics controller 446can receive eye/gaze information from eye tracker 434 and control thelight that enters the user's eye according to the eye/gaze information.In some implementations, optics controller 446 can receive temporalconditions from temporal monitor 436 and control the light that entersthe user's eye according to temporal conditions.

In some implementations, optics controller 446 can receive output fromeye model(s) 440 (e.g., control parameters) and control the light thatenters the user's eye according to the output. In some implementations,optics controller 446 can receive output from eye model(s) 440 andcontrol the light that enters the user's eye according to the output andtracked dynamic state parameters for the user (e.g., pupil location,pupil size, etc.). Implementations of eye model(s) 440 can bepersonalized to configure the XR system to achieve optical lightconditions/optical parameters for the individual eyecharacteristics/tracked dynamic state parameters for a user.Implementations of optics controller 446 can control the lightconditions presented to a user by the XR system to achieve theseoptimizations.

In some implementations, a criteria can be defined, such as a criteriafor an individual monitored visual experience parameter or a combinationof visual experience parameters. When a particular visual experienceparameter related to optics (e.g., optical parameters experienced at theuser's eye, light characteristics, etc.) meets a defined criteria,optics controller 446 can adjust the optics of the light presented to auser by the XR system to vary the particular visual experienceparameter. For example, focal distance criteria can define a thresholdperiod of time for a user to experience a same focal distance. When amonitored profile for the user indicates that the focal distancecriteria has been met, optics controller 446 can adjust the focaldistance to provide the user a varied experience. Other monitored visualexperience parameters can have defined criteria that trigger adjustmentsto light/optics presented to a user to vary the visual experienceparameters. Additional details on optics controller 446 are providedbelow in relation to blocks 806 and 818 of FIG. 8 and blocks 912 and 914of FIG. 9 , and block 1010 of FIG. 10 , and blocks 1114 and 1116 of FIG.11 .

Implementations adjust the light that enters a user's eyes via ahead-mounted display (HMD) that provides the user an immersive XRexperience. FIG. 5 is a conceptual diagram illustrating light entering auser's eye via an XR system. Diagram 500 includes HMD 502, user 504, andlight 506. In some implementations, HMD 502 comprises a display thatemits light 506 to the eyes of user 504 to generate an immersive XRenvironment experienced by the user, such as a VR environment (or AR/MRenvironment generated using captured video). In some implementations,HMD 502 is configured such that real-world light passes through thedevice and additional light is generated by the device such that light506 that enters the eyes of user 504 and generates the immersive XRenvironment is a mix of real-world light and light generated by HMD 502,such as an AR or MR environment.

An optics controller and/or eye model at HMD 502 can control the lightthat enters the eyes of user 504 according to user characteristics anddynamic state parameters tracked for user 504 (e.g., trackedcharacteristics of the eyes of user 504). For example, because HMD 502is some distance from the user's eye(s), eye/pupil positioning atdifferent locations in the eye box can impact optical parameters at theuser's eye (e.g., image shell characteristics, or other opticalparameters). The optics controller at HMD 502 can control lenses at HMD502 and/or light emitted from a display at HMD 502 according to thetracked position of a user's eye/pupil and/or output from the eye model.

For example, electric fields can be selectively applied to a GRIN LClens (that is part of HMD 502) to selectively tune portions of the lens,manipulate light that passes through the lens, and control opticalparameters at the user's eye. In another example, switchable componentsof a freeform varifocal optical assembly and/or optical elements of avarifocal optical system with controllable focal power can beselectively controlled to manipulate light that passes through thelens(es), and control optical parameters at the user's eye. In someimplementations, the optics controller at HMD 502 can control light thatenters the eyes of user 504 according to monitored temporal conditions.For example, characteristics of the light emitted from HMD 502 can becontrolled, such as the spectrum, luminance, chromatic balance, andother suitable light characteristics.

FIG. 6 is a conceptual diagram illustrating an artificial realityenvironment with virtual objects. Environment 600 includes virtualobjects 602, 604, 606, 608, and 610, and surfaces 612, 614, and 616. HMD502 of FIG. 5 can generate environment 600 for user 504. Environment 600can be a virtual reality environment, mixed reality environment,augmented reality environment, or any other suitable immersiveenvironment.

Virtual objects 602, 604, 606, 608, and 610 are positioned at differentlocations in the room depicted by environment 600, at different viewingdistances, and having different dimensions. For example, virtual object608 is a two-dimensional panel displayed on surface 614 (e.g., asidewall) while virtual object 602 is a three-dimensional floatingobject. These virtual objects can also be different colors, stationaryor moving, or have other suitable differences. The eyes of user 504 canexperience different optical parameters depending on which virtualobject is in focus for the user. Implementations of an optics controllerat HMD 502 can adjust the virtual object that is displayed to user 504to control the optical parameters at user 504's eye(s).

User 504 can also experience different visual parameters depending onwhich virtual object is displayed to the user. Implementations of acontent controller at HMD 502 can adjust the virtual object that isdisplayed and/or adjust other suitable aspects of environment 600 tovary the visual experience provided to user 504. For example, thespectrum for light experienced by user 504 from the XR system can bemonitored, and it can be determined that this monitored visualexperience parameter meets a criteria defined for the parameters. Inresponse, the colors for one or more of virtual objects 602, 604, 606,608, and 610 and/or surfaces 612, 614, and 616 can be adjusted to varythe user's experience. In other examples, virtual objects can be movedcloser or farther from the user's presence in environment 600, thedimension or shape of virtual objects can be adjusted, or other suitablevisual experience parameters can be varied.

FIG. 7 is a conceptual diagram illustrating variations of opticalconditions at a user's eye. Diagram 700 includes eyes 702, 704, and 706,and image shells 708, 710, and 712, where the arrows illustrate thefocal distance for portions of the image shells. Image shell 708 depictsa conventional out-of-focus image shell. For example, image shell 708does not fall on almost any part of the retinal plane of eye 702. Imageshells 710 and 712 depict alterations to image shell 708 to correct forthe distortion. However, the curvature of image shells 710 and 712 maystill produce unclear images. In particular, image shells 710 and 712may produce out of focus peripheral images because portions of the imageshell do not align with the retinal plane edges of eyes 704 and 706.Implementations manipulate light that enters a user's eye to controloptical parameters at the user's eye, such as characteristics of theimage shell (e.g., curvature, centration, etc.), characteristics of theperiphery, the optical axis, focal distance at eccentricity, modulationtransfer function (MTF) at eccentricity, myopic defocus cue inperiphery, contrast/peripheral contrast, Illumination wave shape/dutycycle, and other suitable optical parameters. Example results of suchmanipulations are to produce an image shell that more closely alignswith the entire retinal plane, produce one or more defocus cues in theuser's periphery, or otherwise generate an image shell with propertiesto optimize a user's vision and/or eye biology.

Those skilled in the art will appreciate that the components illustratedin FIGS. 1-7 described above, and in each of the flow diagrams discussedbelow, may be altered in a variety of ways. For example, the order ofthe logic may be rearranged, substeps may be performed in parallel,illustrated logic may be omitted, other logic may be included, etc. Insome implementations, one or more of the components described above canexecute one or more of the processes described below.

FIG. 8 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using an XR system and tracked userconditions. In some implementations, process 800 can be used tomanipulate the light that enters a user's eye(s) and control opticalparameters at the user's eye(s). Process 800 can be triggered when an XRsystem presents an XR environment to a user. Implementations of process800 can be performed by an XR system, such as an HMD.

At block 802, process 800 can display an XR environment to a user. Forexample, an HMD of an XR system can display an immersive XR environmentto a user. The XR system can display the XR environment to a user in anyother suitable manner. At block 804, process 800 can track eye movementby the user. For example, the XR system can include one or more sensorsto track a user's eye movements, such as the pupil positions for one ormore of the user's eyes and/or the user's gaze.

At block 806, process 800 can determine whether optical adjustmentshould be performed based on the tracked eye/gaze movement. For example,it can be determined that optical adjustment should be performed forcertain pupil positions in the eye box (e.g., edges of the eye box,other suitable positions) and/or certain eye gaze movements/positions.In some implementations, when the pupil moves to an edge of the eye box,the optics for light entering the user's eye via the pupil can beimpacted. Accordingly, it can be determined that optical adjustmentshould be performed when eye tracking detects certain eye/pupil/gazeconditions. When it is determined that optical adjustment should beperformed, process 800 progresses to block 808. When it is determinedthat optical adjustment should not be performed, process 800 loops backto block 802, where the XR environment continues to be displayed to theuser.

At block 808, process 800 adjusts optics to control optical parametersat the user's eyes. For example, one or more devices at the XR system(e.g., lenses) can be controlled to adjust the light entering the user'seye. In another example, light emitted by the XR system can becontrolled to adjust the light entering the user's eye. In someimplementations, adjusting the light entering the user's eye controlsone or more of a focal distance of one or more objects displayed to auser, one or more defocus cues created at the user's eye, a wave shapefor the light entering the user's eye, a chromatic balance of the lightentering the user's eye, luminance, a field of view displayed to theuser, an image shell created at the user's eye, a curvature of the imageshell created at the user's eye, a peripheral curvature of the imageshell created at the user's eye, a centration of curvature of the imageshell created at the user's eye, peripheral myopic defocus cues in theuser's periphery, peripheral contrast, or any combination thereof.

In an example, controlling an XR system to adjust light entering theuser's eye can include selectively applying an electric field to a GRINLC lens to selectively tune portions of the lens, manipulate light thatpasses through the lens, and control optical parameters at the user'seye. In another example, switchable components of a freeform varifocaloptical assembly and/or optical elements of a varifocal optical systemwith controllable focal power can be selectively controlled tomanipulate light that passes through the lens(es), and control opticalparameters at the user's eye.

In some implementations, the optical axis is adjusted according to thetracked eye location/eye gaze. For example, the optical axis can becontrolled to maintain a consistent image shell curvature on a user'sretina in the presence of user eye/gaze movement. In someimplementations, a “swimming effect” caused by optical components of theXR system can be mitigated by the adjustments. For example, a swimmingeffect can cause objects to appear distorted when a user's head/eye(s)move as light is entering the user's eye(s) via a different portion ofthe optical system/lens. The swimming effect can be mitigated byimplementations, for example by controlling the focal distance (atcenter of fixation), the image shell curvature, and/or the image shellcentration.

FIG. 9 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using an XR system according to monitoredtemporal conditions. In some implementations, process 900 can be used tomanipulate the light that enters a user's eye(s) and to control opticalparameters at the user's eye(s). Process 900 can be triggered when an XRsystem presents an XR environment to a user. Implementations of process900 can be performed by an XR system, such as an HMD.

At block 902, process 900 can track temporal conditions for a user.Example temporal conditions that are tracked include time of day, day ofweek, day of the month, season of the year, user sleep cycle, useractivity level, user activity type, or any other suitable temporalcondition. In some implementations, one or more devices (e.g., awearable device with a sensor) can perform tracking functions andtransmit one or more of the tracked temporal conditions to the XRsystem.

At block 904, process 900 can determine whether a variance criteria ismet by the temporal conditions. An example variance criteria can includetiming parameters related to a user's sleep cycle. The light exposed toa user can impact the user's circadian rhythms. Accordingly, when thetime of day is within a threshold duration from a user's sleep cycle(e.g., bedtime), the variance criteria can be met.

In another example, the variance criteria can be defined times of theday. A user may be impacted by luminance, spectrum, chromatic balance,and/or other light characteristics from the XR system (e.g., light thatenters the user's eye via the XR system) in different ways according tothe time of day and/or amount of natural light exposed to the user.Accordingly, the variance criteria can be met during predefined times ofday and/or predefined times of day during predefined seasons.

When the variance criteria is met, process 900 can progress to block908. When the variance criteria is not met, process 900 can progress toblock 906. At block 906, process 900 can display an XR environment tothe user. For example, an HMD of an XR system can display an immersiveXR environment to a user. The XR system can display the XR environmentto a user in any other suitable manner.

At block 908, process 900 can determine whether a display adjustmentshould be performed. For example, when the variance criteria is met, oneor more adjustments can be made to control the light the enters theuser's eye(s) via the XR system. In some implementations, the light thatenters the user's eye(s) via the XR system is emitted by the XR system.Depending on the tracked temporal conditions and/or the met variancecriteria, one or more display adjustments can be performed.

For example, when the variance criteria that is met is a time of dayproximate to the user's sleep cycle, it can be determined that a displayadjustment should be performed to mitigate an impact on the user'scircadian rhythm. In another example, when the variance criteria met isa predefined time of day, it can be determined that a display adjustmentshould be performed to optimize characteristics of light emitted by theXR system for the time of day.

When it is determined that a display adjustment should be performed,process 900 progresses to block 910. When it is determined that adisplay adjustment should not be performed, process 900 progresses toblock 912. At block 910, process 900 can adjust a display presented to auser according to the tracked temporal conditions. For example,characteristics of light emitted by the XR system, such as theluminance, spectrum, chromatic balance, any other suitable lightcharacteristics can be adjusted according to the temporal conditions.

At block 912, process 900 can determine whether a lens adjustment shouldbe performed. For example, a lens adjustment can control characteristicof image shell generated at a user's eye (e.g., image shell curvature,image shell centration, optical axis, etc.), characteristics of theoptical parameters at a user's periphery (e.g., peripheral defocus cues,peripheral contrast, peripheral curvature), and other suitable opticalparameters at the user's eye. These optical parameters can impact auser's vision, eye biology, and/or eye fatigue. Accordingly, lensadjustments can be performed to improve the impact the XR system has onthe user's eye. In some implementations, it can be determined that alens adjustment should be performed according to one or more definedtimes of day.

When it is determined that a lens adjustment should be performed,process 900 progresses to block 914. When it is determined that a lensadjustment should not be performed, process 900 progresses to block 906,where the XR environment is displayed to a user according to anyperformed display adjustment (or any other suitable adjustment).

At block 914, process 900 can adjust one or more lenses of the XR systemaccording to the tracked temporal conditions. For example, a lensadjustment can control one or more of focal distance, MTF (ateccentricity), field of view, image shell curvature, image shellcentration, peripheral defocus cues, peripheral contrast, or any othersuitable optical parameters at the user's eye. In some implementations,the curvature of an image shell, peripheral defocus cues, and/or MTF canbe controlled according to the monitored temporal conditions, such asthe time of day.

In an example, adjusting one or more lenses of the XR system can includeselectively applying an electric field to a GRIN LC lens to selectivelytune portions of the lens, manipulate light that passes through thelens, and control optical parameters at the user's eye. In anotherexample, switchable components of a freeform varifocal optical assemblyand/or optical elements of a varifocal optical system with controllablefocal power can be selectively adjusted to manipulate light that passesthrough the lens(es), and control optical parameters at the user's eye.After block 914, process 900 can progress to block 906 and display theXR environment to the user using any suitable adjustments performed atblocks 910 and 914.

FIG. 10 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling opticalparameters at a user's eye using a personalized eye model. In someimplementations, process 1000 can be used to manipulate the light thatenters a user's eye(s) and control optical parameters at the user'seye(s). Implementations of process 1000 can be performed by an XRsystem, such as an HMD.

At block 1002, process 1000 can receive characteristics of a user's eye.Example user eye characteristics include visual optics/refractive error(e.g., spherical, across field), axial length, choroidal thickness,ocular curvature, phoria at near/far, and the like. In someimplementations, other user characteristics are also received with theuser eye characteristics, such as user age, medical history, and othersuitable user information. In some implementations, user characteristicsinclude spherical refraction or degree of myopia (e.g., in diopters) or‘SR’, degree of accommodation (e.g., in diopters) or ‘Acc’, age (e.g.,in years).

In some cases, eye characteristics can be determined from user recordse.g., medical records, eye prescriptions, etc., that include a user'seye information. In additional cases, one or more sensors (e.g.,cameras) can capture data for the user's eyes (e.g., under differentlighting conditions/eye circumstances, using different light types,using depth sensors, etc.), and the captured sensor data can beprocessed to generate a user's eye characteristics. In further cases,one or more eye tests can be administered, and the user's eyecharacteristics can be included in the eye test results.

At block 1004, an eye model can be generated for the user. In somecases, process 1000 can use the eye characteristics to determinephysical dimensions of the eye and generate a corresponding 3D model.Simulated light can then be applied to this 3D model, according to knowneffects of light on human eyes, to determine optimized controlparameters for the user's eyes. In some cases, the eye characteristicscan be applied to a machine learning model trained to take eyecharacteristics (which may include the sensor data and/or higher orderresults from such sensor data such as the aforementioned 3D eye model)and output optimized optical control parameters. For example, recordedmedical data across a set of users who have logged different kinds oflight exposure can be used as training data for such a machine learningmodel. Thus, the generated eye model can take as input the received usercharacteristics (which may be converted into a 3D model) and outputoptimized control parameters that support optimal optical parametersexperienced at the user's eye(s). Such a personalized eye model can be a3D model with corresponding physics rules, a regression model, a machinelearning model, an algorithm, a lookup table, and/or any other suitablemodel or combination of models.

In some implementations, the eye model can be generated using model datathat represents healthy and/or optimized user experiences. For example,the model data can include sets of data that match favorable valueranges for one or more optical parameters experienced at the user's eye(e.g., an image shell created at the user's eye, a curvature of theimage shell created at the user's eye, a peripheral curvature of theimage shell created at the user's eye, a centration of curvature of theimage shell created at the user's eye, peripheral myopic defocus cues inthe user's periphery, peripheral contrast, and the like) to differentsets of user characteristics. Implementations of the generated eye modelcan output personalized control parameter values for one or more ofthese optical parameters experienced at the user's eye.

In some implementations, a personalized eye model can simulate opticalparameters at the eye according to dynamic state parameters tracked forthe user. For example, an eye model can receive as input one or morestatic eye characteristics (e.g., spherical refraction, age, etc.) andone or more tracked dynamic eye parameters (e.g., accommodation state).The definitions of one or more eye model elements (e.g., anteriorcornea, posterior cornea, pupil, anterior lens, posterior, retina, anyother suitable eye model element, or any combination thereof) can bebased on static eye characteristics, dynamic eye parameters (e.g.,accommodation state), or any combination thereof. For example, when aneye model element definition is based on a dynamic eye parameter, thevalues used to simulate optical parameters according to this eye modelelement are adjusted when user eye tracking indicates a change in thedynamic eye parameter.

In some implementations, a personalized eye model can map usercharacteristics and dynamic state parameters to optical parametercontrols. For example, model data can include sets of data that matchfavorable value ranges for one or more optical parameters experienced atthe user's eye to different sets of user characteristics and differentsets of dynamic state parameters (e.g., pupil size, pupil position ineye box, gaze vector/fixation distance, accommodative state, vergencestate/distance, time of day, recent time of fixation at distance,background visual distance/intensity, etc.). Implementations of thegenerated eye model can output personalized control parameter values forone or more of these optical parameters experienced at the user's eyeaccording to the monitored dynamic state parameters for a user (e.g.,dynamic state parameters monitored while a user in immersed in an XRenvironment).

At block 1006, process 1000 can initiate an XR environment for the user.For example, the user can interact with the XR system in a variety ofmodes. An XR environment can be initiated for the user when the userdons an XR device to take an action through such a device (e.g., entersan area/moves to a location related to an XR environment). In someimplementations, an HMD of an XR system can display an immersive XRenvironment to the user.

At block 1008, process 1000 can track dynamic parameters for the user.For example, the XR system can include one or more sensors to track thedynamic state parameters for the user while the user is immersed in theXR environment. Example dynamic state parameters that can be trackedinclude pupil size, pupil position in eye box, gaze vector/fixationdistance, accommodative state, vergence state/distance, time of day,recent time of fixation at distance, background visualdistance/intensity (e.g., for AR/MR), and other suitable dynamic stateparameters. In various cases depending on the configuration of the eyemodel, such tracked dynamic parameters can be used to update thegenerated eye model for the user and/or as inputs to the generated eyemodel.

At block 1010, process 1000 can adjust optics to control opticalparameters at the user's eyes according to the tracked dynamic stateparameters and output from the eye model. For example, the eye model canoutput control parameters for one of more of the optical parameters, andthe optics presented to the user can be adjusted to achieve the controlparameters output by the eye model.

FIG. 11 is a flow diagram illustrating a process used in someimplementations of the present technology for controlling visualparameters by an artificial reality system to vary a user experience. Insome implementations, process 1100 can be used to manipulate the lightthat enters a user's eye(s) and/or adjust content displayed to the userto vary the user's experience. Process 1100 can be triggered when an XRsystem presents an XR environment to a user. Implementations of process1100 are performed by an XR system, such as an HMD.

At block 1102, process 1100 can display an XR environment to a user. Forexample, an HMD of an XR system can display an immersive or mixedreality XR environment to a user. The XR system can display the XRenvironment to a user in any other suitable manner.

At block 1104, process 1100 can monitor the visual experiences for auser, such as the parameters of the user's visual experiences with theXR system. For example, the monitored visual experience parameters caninclude background characteristics experienced by the user over time(e.g., patterns, colors, distance, aggregated defocus distance, etc.);foreground characteristics experienced by the user over time (e.g.,object shape, dimensions, colors, distance, etc.); luminance, spectrum,and chromatic balance experienced by the user over time; illuminationwave shape/duty cycle experienced by the user over time; contrast (e.g.,contrast polarity) experienced by the user over time; spatial frequencyexperienced by the user over time; longitudinal chromatic aberration(LCA) experienced by the user over time; and other suitable visualexperience parameters related to displayed content. In someimplementations, the visual experience parameters can be monitored bymonitoring the light emitted to the user (e.g., the light displayed inportions of the XR environment by the XR system). For example,characteristics of virtual objects and/or virtual environments displayedto the user can be monitored over time to generate the monitored visualexperience parameters.

In some implementations, the monitored visual experience parameters caninclude optical parameters created at the user's eye by the XR system,such as eye image shell characteristics (e.g., image shell curvature,centration, etc.), periphery characteristics (e.g., periphery curvature,contrast, defocus cues, etc.), focal distance, eye movement,accommodation distances, and any other suitable optical parameters. Forexample, implementation can manipulate light to control opticalparameters at the user's eye(s), and the controlled optical parameterscan be tracked to generate the monitored visual experience parameters.

In some implementations, the monitored optical parameters experienced bythe user/created at the user's eye by the XR system can be simulatedoptical parameters using a personalized eye model. For example, apersonalized eye model for the user can be generated and/or obtained.The personalized eye model can simulate optical parameterscreated/experienced at the user's eye using the light that the XR systemdisplayed to the user.

At block 1106, process 1100 can determine whether one or more parametercriteria have been met by the monitored visual experience parameters. Anexample visual experience criteria can include a threshold amount oftime for a monitored visual experience parameter (e.g., centration ofcurvature of an image shell, field of view, focal distance, etc.) to beconsistently displayed/provided to a user (e.g., displayed with littleor no variation). It can be determined that one or more parametercriteria are met when a given monitored visual experience parametermeets a defined criteria (e.g., threshold amount of time for the givenvisual experience parameter).

When one or more parameter criteria have been met by the monitoredvisual experience parameters, process 1100 can progress to block 1110.When one or more parameter criteria have not been met by the monitoredvisual experience parameters, process 1100 can progress to block 1108.At block 1108, process 1100 can display an XR environment to the user.For example, an HMD of an XR system can display an immersive XRenvironment to a user. The XR system can display the XR environment to auser in any other suitable manner.

At block 1110, process 1100 can determine whether a content adjustmentshould be performed. For example, a content adjustment can adjust adisplayed background, a displayed foreground, a spectrum for emittedlight, a luminance, a distance for an object in focus, a user field ofview, and other suitable content. These content adjustments can impact auser's vision, eye biology, and/or eye fatigue. Accordingly, a contentadjustment can be performed to improve the impact the XR system has onthe user's eye(s).

When it is determined that a content adjustment should be performed,process 1100 progresses to block 1112. When it is determined that acontent adjustment should not be performed, process 1100 progresses toblock 1114. At block 1112, process 1100 can adjust the contentpresented/displayed to a user according to the monitored visualexperience parameter(s). In some implementations, when a met criteriacorresponds to a monitored visual experience parameter for the field ofview presented to the user, it can be determined that the content shouldbe adjusted to vary the field of view (e.g., grow or shrink the field ofview). In another example, when the met criteria corresponds to amonitored visual experience parameter for a contrast polarity presentedto the user, it can be determined that the content should be adjusted tovary the contrast polarity (e.g., generate a larger or smaller chromaticcontrast between a foreground and background). In another example, whenthe met criteria corresponds to a monitored visual experience parameterfor a spatial frequency experienced by the user, it can be determinedthat the content should be adjusted to vary the display of one or moreobjects, such as by moving the object closer or farther from the userpresence in the XR environment, growing or shrinking a size of theobject, or altering the display of the object in any other manner tovary the spatial frequency experienced by the user. Implementations ofthe XR system can alter content by instructing a software application(e.g., that implements the XR environment) to adjust the relevantcontent. Any other suitable technique to alter contentpresented/displayed to the user can be implemented.

At block 1114, process 1100 can determine whether an optics adjustmentshould be performed. For example, an optics adjustment can controlcharacteristics of an image shell generated at a user's eye (e.g., imageshell curvature, image shell centration, optical axis, etc.),characteristics of the optical parameters at a user's periphery (e.g.,peripheral defocus cues, peripheral contrast, peripheral curvature), andother suitable optical parameters at the user's eye. These opticalparameters can impact a user's vision, eye biology, and/or eye fatigue.Accordingly, an optics adjustment can be performed to improve the impactthe XR system has on the user's eye(s).

In some implementations, when the met criteria corresponds to amonitored visual experience parameter for image shell centration at theuser's eye, it can be determined that the optics should be adjusted tovary the image shell centration. In another example, when the metcriteria corresponds to a monitored visual experience parameter for aperipheral curvature at the user's eye, it can be determined that theoptics should be adjusted to vary the peripheral curvature. In anotherexample, when the met criteria corresponds to a monitored visualexperience parameter for a focus distance at the user's eye, it can bedetermined that the optics should be adjusted to vary the focusdistance.

When it is determined that an optics adjustment should be performed,process 1100 progresses to block 1116. When it is determined that anoptics adjustment should not be performed, process 1100 progresses toblock 1108, where the XR environment is displayed to a user according toany performed content adjustment (or any other suitable adjustment).

At block 1116, process 1100 can adjust one or more lenses of the XRsystem to adjust the optics presented/displayed to a user. In someimplementations, a lens adjustment can control one or more of focaldistance, MTF (at eccentricity), field of view, image shell curvature,image shell centration, peripheral defocus cues, peripheral contrast, orany other suitable optical parameters at the user's eye. For example: a)when the met criteria corresponds to a monitored visual experienceparameter for image shell centration at the user's eye, one or morelenses can be controlled to vary the image shell centration; b) when themet criteria corresponds to a monitored visual experience parameter fora peripheral curvature at the user's eye, one or more lenses can beadjusted to vary the peripheral curvature; and/or c) when the metcriteria corresponds to a monitored visual experience parameter for afocus distance at the user's eye, one or more lenses can be adjusted tovary the focus distance.

In an example, controlling an XR system to adjust light entering theuser's eye can include selectively applying an electric field to a GRINLC lens to selectively tune portions of the lens, manipulate light thatpasses through the lens, and control optical parameters at the user'seye. In another example, switchable components of a freeform varifocaloptical assembly and/or optical elements of a varifocal optical systemwith controllable focal power can be selectively controlled tomanipulate light that passes through the lens(es) and control opticalparameters at the user's eye. After block 1116, process 1100 canprogress to block 1108 and display the XR environment to the user usingany suitable adjustments performed at blocks 1112 and 1116

In some implementations, output control parameters can control one ormore devices at the XR system (e.g., lenses) to adjust the lightentering the user's eye. In another example, the output controlparameters can control light emitted by the XR system to adjust thelight entering the user's eye. In some implementations, the outputcontrol parameters can control adjustments for the light entering theuser's eye, such as to control/adjust a focal distance of one or moreobjects, one or more defocus cues created at the user's eye, a chromaticbalance of the light entering the user's eye, luminance, a field of viewdisplayed to the user, an image shell created at the user's eye, acurvature of the image shell created at the user's eye, a peripheralcurvature of the image shell created at the user's eye, a centration ofcurvature of the image shell created at the user's eye, peripheralmyopic defocus cues in the user's periphery, peripheral contrast, or anycombination thereof.

In an example, controlling an XR system to adjust light entering theuser's eye can include selectively applying an electric field to a GRINLC lens to selectively tune portions of the lens, manipulate light thatpasses through the lens, and control optical parameters at the user'seye. In another example, switchable components of a freeform varifocaloptical assembly and/or optical elements of a varifocal optical systemwith controllable focal power can be selectively controlled tomanipulate light that passes through the lens(es), and control opticalparameters at the user's eye.

In some implementations, an eye model for a user can be received (e.g.,rather than generated), and the received eye model can be used toprovide a personalized XR experience for the user. For example, trackeddynamic state parameters can be input to the received model, controlparameters can be output by the received model, and optics presented tothe user by the XR system can be adjusted according to the outputcontrol parameters.

In some implementations, the optical axis is adjusted according to thecontrol parameters output by the eye model. For example, the opticalaxis can be controlled to maintain a consistent image shell curvature ona user's retina in the presence of user eye/gaze movement. In someimplementations, a “swimming effect” caused by optical components of theXR system can be mitigated by the adjustments. For example, a swimmingeffect can cause objects to appear distorted when a user's head/eye(s)move as light is entering the user's eye(s) via a different portion ofthe optical system/lens. The swimming effect can be mitigated byimplementations, for example by controlling the focal distance (atcenter of fixation), the image shell curvature, and/or the image shellcentration.

Implementations of eye model(s) simulate optical parameters experiencedat a user's eye from light that enters the user's eye. Such eye model(s)include techniques to simulate how light controlled by XR device(s) isexperienced at the user's eye. FIG. 12 is a diagram illustrating a modelof the optical parameters of light output from a display device. Model1200 includes XR system 1202, light source 1204, and output light 1206.In some implementations, light source 1104 can be part of XR system1202. In another example, light source 1204 can be a simulatedreal-world light source that emits light that passes through XR system1202 (e.g., one or more lenses of the system). Model 1200 models thecharacteristics of output light 1206 that is output from XR system 1202.

FIG. 13 is a diagram illustrating another model of the opticalparameters of light output from another display device. Model 1300includes XR system 1302, light source 1304, and output light 1306. Insome implementations, light source 1304 can be part of XR system 1302.In another example, light source 1304 can be a simulated real-worldlight source that emits light that passes through XR system 1302 (e.g.,one or more lenses of the system). Model 1300 models the characteristicsof output light 1306 that is output from XR system 1302.

In some implementations, the light output from XR systems can be modeledin combination with eye model(s) disclosed herein. FIG. 14 is a diagramillustrating a model of the optical parameters of light output from adisplay device in combination with an eye model. Model 1400 includes XRsystem lens 1402, light source 1404, output light 1406 and eye model1408. In some implementations, light source 1404 can be part of the XRsystem that provides lens 1402, an external light source (e.g.,simulated real-world light source), or any combination thereof. Model1400 models the optical parameters experienced at eye model 1408 fromoutput light 1406.

FIG. 15 is a diagram illustrating a model of the optical parameters oflight output from another display device in combination with an eyemodel. Model 1500 includes XR system 1502, light source 1504, outputlight 1506 and eye model 1508. In some implementations, light source1504 can be part of XR system 1502, an external light source (e.g.,simulated real-world light source), or any combination thereof. Model1500 models the optical parameters experienced at eye model 1508 fromoutput light 1506 that is output from XR system 1502.

In some implementations, models 1200, 1300, 1400, and 1500 can be 3Dmodels configured for use with modeling software (e.g., Zemax models).Eye models 1408 and 1508 can include stacked/combined model elementsthat comprise the human eye, such as pupil elements, cornea elements,retina elements, and the like. Each model element can include definedcharacteristics that control the way light is simulated at the eyemodel(s), such as how light is altered when passing through the modelelement.

Example model elements of the eye model(s) include an anterior cornea(e.g., defined by a radius, thickness, asphericity, refractive index,etc.), posterior cornea (e.g., defined by a radius, thickness,asphericity, refractive index, etc.), pupil (e.g., defined by asemidiameter), anterior lens (e.g., defined by a radius, thickness,asphericity, refractive index, etc.), posterior lens (e.g., defined by aradius, thickness, asphericity, refractive index, etc.), retina (e.g.,defined by a radio, asphericity, etc.), and any other suitable eye modelelement. In some implementations, the eye model elements can be definedrelative to characteristics of the user's eye, such as sphericalrefraction or degree of myopia (e.g., in diopters) or ‘SR’, degree ofaccommodation (e.g., in diopters) or ‘Acc’, age (e.g., in years), andany other suitable characteristics.

The following represents definitions (e.g., values, mathematicalexpressions) for eye model elements based on user eye characteristics(e.g., SR, Acc, age) in some implementations:

-   -   Anterior Cornea        -   Radius (mm)            -   R=7.77+0.022*SR        -   Asphericity            -   Q=−0.24+0.003*Age        -   Thickness            -   d=0.55        -   Refractive Index            -   n=1.361594+6.009687*10⁻³-6.760760*10⁻⁴*λ⁻⁴+5.908450*10⁻⁵*λ⁻⁶            -   Simplified implementation: n=1.376    -   Posterior Cornea        -   Radius (mm)            -   R=6.4        -   Asphericity            -   Q=−0.1-0.007*Age        -   Thickness            -   d=3.15-0.05*ln(Acc+1)        -   Refractive Index (Aqueous Humour)            -   n=1.323016+6.077158*10⁻³*λ⁻²-7.069706*10⁻⁴*λ⁻⁴+6.154303*10⁻⁵*λ⁻⁶            -   Simplified implementation: n=1.3374    -   Pupil        -   Semidiameter (mm)            -   Sd=3    -   Anterior Lens        -   Radius (mm)            -   R=12.9-0.057*Age−1.75*ln(Acc+1)        -   Asphericity            -   Q=−5-0.34*ln(Acc+1)        -   Thickness            -   d=3.0+0.2*ln(Acc+1)+0.024*Age        -   Refractive Index (Two Parts)            -   n₁=(1.3807-0.00039*Age+18*10⁻⁵*(10*Acc+Acc²))+0.0652778*Z−0.0226659*Z²−0.0020399(X²+Y²)            -   n₂=(1.4277-0.00039*Age+18*10⁻⁵*(10*Acc+Acc²))+0.0100737*Z²−0.0020399(X²+Y²)    -   Posterior Lens        -   Radius (mm)            -   R=−6.2+0.012*Age+0.2294*ln(Acc+1)        -   Asphericity            -   Q=−2-0.125*ln(Acc+1)        -   Thickness            -   d=16.28-0.299*SR        -   Refractive Index (Vitreous Humour)            -   n=1.322357+5.560240*10⁻³*λ⁻²−5.817391*10⁻⁴*λ⁻⁴+5.036810*10⁻⁵*λ⁻⁶            -   Simplified implementation: n=1.336    -   Retina        -   Radius (mm)            -   R_(x)=12.91-0.094*SR            -   R_(y)=−12.72+0.004*SR        -   Asphericity            -   Q_(x)=0.27+0.026*SR            -   Q_(y)=0.25+0.017*SR

For example, one of more of the eye model elements can be defined ascomponents with a software modeling framework, such as within a Zemaxmaterial catalog (e.g., as a .AGF file). Eye models defined inimplementations can comprise stacks/combinations of these defined eyemodel elements.

Simulated light conditions can be exposed to the eye model(s) thatcomprise the definitions for the eye model elements, and the eyemodel(s) can generate simulated optical parameters experienced at auser's eye. FIG. 16 is a diagram of simulated light exposed to an eyemodel. An example of eye model 1602 comprises a combination/stack ofdefined anterior cornea, posterior cornea, pupil, anterior lens,posterior lens, retina, any other suitable eye model element, and anycombination thereof. For example, the eye model elements of eye model1602 can be defined using the user eye characteristics SR=0, age=25,acc=0, tilt X/Y=0. Simulated light 1604 can be exposed to eye model 1602to simulate optical parameters at the user's eye.

FIG. 17 is a graph of optical characteristics simulated by exposinglight to an eye model. Graphs 1702 and 1704 illustrate the simulatedfield curvature experienced by eye model 1602 when exposed to simulatedlight 1604. The longitudinal chromatic aberration is about 0.35 mm or0.94 D. These values validate that the simulation achieved by eye model1602 matches historic values for longitudinal chromatic aberration.

FIG. 18 is another graph of optical characteristics simulated byexposing light to an eye model. An example of eye model 1802 comprises acombination/stack of defined anterior cornea, posterior cornea, pupil,anterior lens, posterior lens, retina, any other suitable eye modelelement, and any combination thereof. For example, the eye modelelements of eye model 1802 can be defined using the user eyecharacteristics SR=−5 D, age=25, acc=0, tilt X/Y=0. Simulated light 1804can be exposed to eye model 1802 to simulate optical parameters at theuser's eye.

FIGS. 19 and 20 are graphs of optical characteristics simulated byexposing light to another eye model. Graph 1900 of FIG. 19 illustratesthe simulated field curvature in terms of defocus (D) experienced by eyemodel 1802 when exposed to simulated light 1804. The −5 D defocusmatches historic values for the user characteristics used to generateeye model 1802. Graph 2000 of FIG. 20 illustrates the simulated defocus(D) in terms of spherical refraction experienced by eye model 1802 whenexposed to simulated light 1804. The linear relationship between defocusand SR matches historic values, and a longitudinal chromatic aberrationof about 1 D is maintained.

FIGS. 21 and 22 are graphs of optical characteristics simulated byexposing light to eye model variations. Graph 2100 of FIG. 21illustrates simulated defocus (D) in terms of different sphericalrefraction values (e.g., simulated using eye models defined by a rangeof spherical refraction values). The linear relationship between defocusand SR matches historic values, and a longitudinal chromatic aberrationof about 1 D is maintained. Graph 2200 of FIG. 22 illustrates thesimulated defocus (D) in terms of different age values (e.g., simulatedusing eye models defined by a range of age values). The expectedperformance is a maintained focus on-axis across the age range values.The example eye model(s) simulate a less than 0.25 diopter variation foreach wavelength across the age values.

Reference in this specification to “implementations” (e.g., “someimplementations,” “various implementations,” “one implementation,” “animplementation,” etc.) means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation of the disclosure. Theappearances of these phrases in various places in the specification arenot necessarily all referring to the same implementation, nor areseparate or alternative implementations mutually exclusive of otherimplementations. Moreover, various features are described which may beexhibited by some implementations and not by others. Similarly, variousrequirements are described which may be requirements for someimplementations but not for other implementations.

As used herein, being above a threshold means that a value for an itemunder comparison is above a specified other value, that an item undercomparison is among a certain specified number of items with the largestvalue, or that an item under comparison has a value within a specifiedtop percentage value. As used herein, being below a threshold means thata value for an item under comparison is below a specified other value,that an item under comparison is among a certain specified number ofitems with the smallest value, or that an item under comparison has avalue within a specified bottom percentage value. As used herein, beingwithin a threshold means that a value for an item under comparison isbetween two specified other values, that an item under comparison isamong a middle-specified number of items, or that an item undercomparison has a value within a middle-specified percentage range.Relative terms, such as high or unimportant, when not otherwise defined,can be understood as assigning a value and determining how that valuecompares to an established threshold. For example, the phrase “selectinga fast connection” can be understood to mean selecting a connection thathas a value assigned corresponding to its connection speed that is abovea threshold.

As used herein, the word “or” refers to any possible permutation of aset of items. For example, the phrase “A, B, or C” refers to at leastone of A, B, C, or any combination thereof, such as any of: A; B; C; Aand B; A and C; B and C; A, B, and C; or multiple of any item such as Aand A; B, B, and C; A, A, B, C, and C; etc.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Specific embodiments and implementations have been described herein forpurposes of illustration, but various modifications can be made withoutdeviating from the scope of the embodiments and implementations. Thespecific features and acts described above are disclosed as exampleforms of implementing the claims that follow. Accordingly, theembodiments and implementations are not limited except as by theappended claims.

Any patents, patent applications, and other references noted above areincorporated herein by reference. Aspects can be modified, if necessary,to employ the systems, functions, and concepts of the various referencesdescribed above to provide yet further implementations. If statements orsubject matter in a document incorporated by reference conflicts withstatements or subject matter of this application, then this applicationshall control.

1. A method for automatically controlling visual parameters by anartificial reality (XR) system according to a tracked user experience,the method comprising: displaying, by the XR system, an XR environmentto a user; monitoring, over a time period, visual parameters experiencedby the user based on the displayed XR environment; comparing themonitored visual parameters to one or more criteria; and controlling, inresponse to the monitored visual parameters meeting the one or morecriteria, one or more characteristics of light that enters the user'sone or more eyes via the XR system and/or content within the XRenvironment.
 2. The method of claim 1, wherein the monitored visualparameters experienced by the user over the time period comprise one ormore of focal distance, fixation distance, background visual distance,background visual intensity, aggregated defocus distance of background,luminance, visual activity, eye movement, accommodation distances, orany combination thereof.
 3. The method of claim 1, wherein the monitoredvisual parameters experienced by the user over the time period comprisechromatic balance, illumination wave shape or duty cycle, contrastpolarity, spatial frequency, longitudinal chromatic aberration (LCA), orany combination thereof.
 4. The method of claim 1, wherein thecontrolling the characteristics of light that enters the user's one ormore eyes controls at least one of a focal distance of one or moreobjects displayed to a user, a wave shape for the light entering theuser's eye, a chromatic balance of the light entering the user's eye, apolarity of the light entering the user's eye, luminance, a field ofview displayed to the user, or any combination thereof.
 5. The method ofclaim 1, wherein the controlling content within the XR environmentcontrols at least one of a focal distance of one or more objectsdisplayed to a user, a size of one or more objects displayed to theuser, a chromatic contrast between a foreground and background of the XRenvironment, or any combination thereof.
 6. The method of claim 1,wherein the one or more criteria comprise a threshold amount of time forat least one of the monitored visual parameters, and the one or morecriteria are met when an amount of time that the user experiences the atleast one monitored visual parameter meets or exceeds the thresholdamount of time.
 7. The method of claim 1, wherein the one or morecriteria comprise a threshold amount of time for a given focal distanceor focal distance range, and the one or more criteria are met when anamount of time that the user experiences the given focal distance orfocal distance range meets or exceeds the threshold amount of time. 8.The method of claim 1, wherein the one or more criteria comprise athreshold amount of time for a given background visual distance orbackground visual distance range, and the one or more criteria are metwhen an amount of time that the user experiences the given backgroundvisual distance or background visual distance range meets or exceeds thethreshold amount of time.
 9. The method of claim 1, further comprising:obtaining an eye model for the user, wherein the eye model is generatedusing at least physical properties of one or more of the user's eyes,and the monitored visual parameters experienced by the user comprisesimulated output from the eye model for the user. 10.-11. (canceled) 12.A computer-readable storage medium storing instructions that, whenexecuted by a computing system, cause the computing system to perform aprocess for automatically controlling visual parameters by an artificialreality system according to a tracked user experience, the processcomprising: displaying, by the artificial reality (XR) system, an XRenvironment to a user; monitoring, over a time period, visual parametersexperienced by the user based on the displayed XR environment; comparingthe monitored visual parameters to one or more criteria; andcontrolling, in response to the monitored visual parameters meeting theone or more criteria, one or more characteristics of light that entersthe user's one or more eyes via the XR system and/or content within theXR environment.
 13. The computer-readable storage medium of claim 12,wherein the monitored visual parameters experienced by the user over thetime period comprise one or more of focal distance, fixation distance,background visual distance, background visual intensity, aggregateddefocus distance of background, luminance, visual activity, eyemovement, accommodation distances, or any combination thereof.
 14. Thecomputer-readable storage medium of claim 12, wherein the monitoredvisual parameters experienced by the user over the time period comprisechromatic balance, illumination wave shape or duty cycle, contrastpolarity, spatial frequency, longitudinal chromatic aberration (LCA), orany combination thereof.
 15. The computer-readable storage medium ofclaim 12, wherein the controlling the characteristics of light thatenters the user's one or more eyes controls at least one of a focaldistance of one or more objects displayed to a user, a wave shape forthe light entering the user's eye, a chromatic balance of the lightentering the user's eye, a polarity of the light entering the user'seye, luminance, a field of view displayed to the user, or anycombination thereof.
 16. The computer-readable storage medium of claim12, wherein the controlling content within the XR environment controlsat least one of a focal distance of one or more objects displayed to auser, a size of one or more objects displayed to the user, a chromaticcontrast between a foreground and background of the XR environment, orany combination thereof.
 17. The computer-readable storage medium ofclaim 12, wherein the one or more criteria comprise a threshold amountof time for at least one of the monitored visual parameters, and the oneor more criteria are met when an amount of time that the userexperiences the at least one monitored visual parameter meets or exceedsthe threshold amount of time.
 18. The computer-readable storage mediumof claim 12, wherein the one or more criteria comprise a thresholdamount of time for a given focal distance or focal distance range, andthe one or more criteria are met when an amount of time that the userexperiences the given focal distance or focal distance range meets orexceeds the threshold amount of time.
 19. The computer-readable storagemedium of claim 12, wherein the one or more criteria comprise athreshold amount of time for a given background visual distance orbackground visual distance range, and the one or more criteria are metwhen an amount of time that the user experiences the given backgroundvisual distance or background visual distance range meets or exceeds thethreshold amount of time.
 20. The computer-readable storage medium ofclaim 12, wherein the process further comprises: obtaining an eye modelfor the user, wherein the eye model is generated using at least physicalproperties of one or more of the user's eyes, and the monitored visualparameters experienced by the user comprise simulated output from theeye model for the user.
 21. An artificial reality system (XR) forautomatically controlling visual parameters according to a tracked userexperience, the system comprising: one or more processors; and one ormore memories storing instructions that, when executed by the one ormore processors, cause the system to perform a process comprising:displaying, by the XR system, an XR environment to a user; monitoring,over a time period, visual parameters experienced by the user based onthe displayed XR environment; comparing the monitored visual parametersto one or more criteria; and controlling, in response to the monitoredvisual parameters meeting the one or more criteria, one or morecharacteristics of light that enters the user's one or more eyes via theXR system and/or content within the XR environment.
 22. The system ofclaim 21, wherein the one or more criteria comprise a threshold amountof time for at least one of the monitored visual parameters, and the oneor more criteria are met when an amount of time that the userexperiences the at least one monitored visual parameter meets or exceedsthe threshold amount of time.